-
Notifications
You must be signed in to change notification settings - Fork 31
✨ Multi cluster controller sharding #74
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
Co-authored-by: Nelo-T. Wallus <[email protected]>
Co-authored-by: Marvin Beckers <[email protected]>
feat: controller sharding
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: zachsmith1 The full list of commands accepted by this bot can be found here.
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
Welcome @zachsmith1! |
this looks amazing 🚀 with this implementation, is it possible to shard individual clusters too or is it per cluster? |
This is just for clusters but I think there is another community project we could leverage for the intra cluster sharding |
What
Deterministic sharding via HRW (rendezvous) over live peers.
Peer leases mcr-peer-* (membership/weights) + shard leases mcr-shard- (per-cluster fencing, single writer).
Clean handoff: watches detach/re-attach on ownership change.
Rebalance on scale up/down.
Why
Prevent double reconciles & cold-start stampede.
Balanced, deterministic ownership with fast failover.
How to try
Follow the README (examples/sharded-namespace) for build/deploy/observe steps.
Changes
Manager: HRW + per-cluster Lease fencing, peer/fence prefix split.
Controller: per-cluster engagement context, re-engage fix.
Source: removable handler, re-register on new ctx.